Learning Gaussian Graphical Models with Observed or Latent FVSs

Gaussian Graphical Models (GGMs) or Gauss Markov random fields are widelyused in many applications, and the trade-off between the modeling capacity andthe efficiency of learning and inference has been an important research prob-lem. In this paper, we study the family of GGMs with small feedback vertexsets (FVSs), where an FVS is a set of nodes whose removal breaks all the cycles.Exact inference such as computing the marginal distributions and the partitionfunction has complexity O(kn) using message-passing algorithms, where k isthe size of the FVS, and n is the total number of nodes. We propose efficientstructure learning algorithms for two cases: 1) All nodes are observed, which isuseful in modeling social or flight networks where the FVS nodes often corre-spond to a small number of highly influential nodes, or hubs, while the rest ofthe networks is modeled by a tree. Regardless of the maximum degree, withoutknowing the full graph structure, we can exactly compute the maximum likelihoodestimate with complexity O(kn + n log n) if the FVS is known or in polyno-mial time if the FVS is unknown but has bounded size. 2) The FVS nodes arelatent variables, where structure learning is equivalent to decomposing an inversecovariance matrix (exactly or approximately) into the sum of a tree-structured ma-trix and a low-rank matrix. By incorporating efficient inference into the learningsteps, we can obtain a learning algorithm using alternating low-rank correctionswith complexity O(kn + n log n) per iteration. We perform experiments usingboth synthetic data as well as real data of flight delays to demonstrate the modelingcapacity with FVSs of various sizes.